In this notebook, we will introdcue a few R packages that can be handy to manipulate, summarize and plot the data coming from the PinAPL-py pipeline. Importantly, the methodologies presented here can be applied and expanded to any reasonnably sized dataset.

The packges used are loaded using the following set of commands. This assumes the packages have been installed using the install.package command or the Rstudio Tools>Install Packages menu.

library(dplyr) # data wrangling and dataframe manipulation
library(tidyr) # data wrangling and dataframe manipulation
library(reshape2) # conversion between long and wide dataframe formats
library(ineq) # calculating gini coefficient (and others)
library(ggplot2) # verstile plotting package
library(pheatmap) # clustering and plotting pretty heatmaps

In order to best benefit from this practise and use case, one needs to be familiar with R data structure and manipulation, mostly differentating vector, matrix and dataframes. A primer on the different data types (numeric, integer, character, etc) is also useful.

The data used in this workshop corresponds to a GECKO screen in a cancer cell line, looking for enrichment of shRNA in cells treated with a genotoxic drug for 1h. 4 libraries from each Baseline (T0), Treated (T3_T) and Untreated (T3_U) were sequenced. In the first part we will study the count files, in a second section the significance analysis.

Data Import and Formatting

The pipeline places the count data in different folders. While there is a way to have R crawl through mulitple folders and import the data, we will assume here, for convenience, that the counts files have been placed in the same “counts” folder and renamed after the library ID. We will import the non-normalized counts.

FiRst we write a function that takes the file name (and path), import the content into a dataframe, names the columns, and add a column containing the name of the file.

importCounts<-function(x){
  f<-read.delim(x,header=FALSE)
  colnames(f)<-c("sgRNA","gene", "value")
  f$Library<-x
  return(f)
}

Now we apply this function to a list of files in the folder.

files <- list.files(path="./counts",pattern=".tsv",full.names = TRUE)
CountSg<-sapply(files,importCounts,simplify=FALSE)

We get a list of dataframes, which we collapse into one giant dataframe to which we remove the rownames, since they are not relevant.

CountSg<-do.call(rbind,CountSg)
rownames(CountSg)<-NULL

We can now edit the dataframe and separate the library fields into subfields using the function tidyr::separate function

CountSg<-separate(CountSg, Library, sep="[./]",into=c("pre1","pre2","folder","Library","ext1","ext2"),remove=TRUE)

and we keep only the column we care about using dplyr::select

CountSg<-select(CountSg,sgRNA,gene,value,Library)

We have now a large dataframe containing all the counts data

Summary Statistics and Normalization

Here we will use mostly functions from the dplyr package. for example to determine the total number of counts in each library

totalCounts<-CountSg %>% group_by(Library) %>% summarize(Total=sum(value))
totalCounts

We can therefore normalized the counts to the total number of counts in each library and multiply by 1E6 to get an RPM (read per million) unit. For this we can use the function dplyr::mutate, which can assign tghe results of an operation to a new column. But before we specigy the grouping variable.

CountSg<-CountSg %>% group_by(Library) %>% mutate(RPM=value*1E6/sum(value))

We can now verify that all the libraries have the same sum of RPM and can be compared

CountSg %>% group_by(Library) %>% summarize(Total=sum(RPM))

plotting cumulative distribution of counts

Let’s now look at the distribution of RPM in each library. For this, we can use the ggplot2 package to plot the cumulative distribution function. Which is similar to the Lorenz plot generated by PinAPL-Py. Let’s start with one library.

let’s start with one library using the dplyr::filter command

C1<-filter(CountSg,Library=="Control_1")

A typical ggplot command starts with the ggplot fucntion, which specifies the dataframe and the variable to use in the “aesthetic” (x, y, colors, sizes, etc, etc). This function is them modified by adding the type of plot desired, here stat_ecdf(). The RPM values are following an exponential distribution (lots of low covered sgRNA), we will therefore plot their log10. In order to also represent the sgRNA that are not covered in the library (RPM=0), we will add a small number to all RPM. Finally, we specific the size of the plot (in inches) in the header.

ggplot(C1,aes(log10(RPM+0.01)))+stat_ecdf()

We can now change a bit the labels and the size of the font to make it prettier

ggplot(C1,aes(log10(RPM+0.01)))+stat_ecdf()+
  ylab("Fraction of sgRNA")+
  xlab("log10(RPM)")+
  theme(text=element_text(size=20))

and now we can plot all libraies on the same plot, by starting from the full datafrane and adding a color variable to the aesthetic, (adn adjusting the)and adjusting panel wiodth to make room for the legend)

ggplot(CountSg,aes(log10(RPM+0.01),col=Library))+stat_ecdf()+
  ylab("Fraction of sgRNA")+
  xlab("log10(RPM)")+
  theme(text=element_text(size=20))

Now, this is too many libraries to visualize on one plot, we shoudl be able to separate them by timepoint and condition. We need to add another field to the dataframe. For this we will use the function dplyr::mutate seen before and perfrom a logical test based on the name of the library using grepl. grepl returns TRUE if the queries string is present in the tested string, FALSE otherwise. The function ifelse will assign the type to the type variable, as a result of this test.

CountSg<-mutate(CountSg,type=ifelse(grepl("T0",Library),"Baseline","T3"))
CountSg<-mutate(CountSg,type=ifelse(grepl("T3_T",Library),"treated_T3",type))
CountSg<-mutate(CountSg,type=ifelse(grepl("T3_U",Library),"untreated_T3",type))

we can verify the new field, by listing the unique columns

CountSg %>% select(Library,type) %>% unique()

and we can now use the fucntion facet_wrap in ggplot to separate each type on a separate panel, increaseing the width (resp heigth) to make room if we display them in a row (resp. column)

ggplot(CountSg,aes(log10(RPM+0.01),col=Library))+stat_ecdf()+
  ylab("Fraction of sgRNA")+
  xlab("log10(RPM)")+
  theme(text=element_text(size=20))+
  facet_wrap(~type,ncol=3)

We can clearly see that some selection happened in the treated sample. Let’s calculte the gini coefficient for each library. this is provided by the function ineq::ineq

gini<-CountSg %>% group_by(Library,type) %>% summarize(gini=ineq(RPM,type="Gini"))
gini

Let’s plot the corresponding bar graph using the geom_bar function. Beause the value to plot is directly given by the gini field and that no math or calculations are needed, we need to specific stat=“identity”

ggplot(gini,aes(Library,gini,fill=type))+geom_bar(stat="identity")+
 ylab("gini")+
  xlab("Libraries")+
  theme(text=element_text(size=14))

Now this is not pretty, we can rotate the x axis labels in the theme variable. But before we need to change the levels of the Name and type variable so that they are not plotted in alphabetical order.

gini$Library<-factor(gini$Library,levels=c("I1R1T0_1","I1R2T0_1","I2R1T0_1","I2R2T0_1","I1R1T3_U","I1R2T3_U","I2R1T3_U","I2R2T3_U","I1R1T3_T","I1R2T3_T","I2R1T3_T","I2R2T3_T"))

Now we can replot, rotatign the x axiss

ggplot(gini,aes(Library,gini,fill=type))+geom_bar(stat="identity")+
 ylab("gini")+
  xlab("Libraries")+
  theme(text=element_text(size=14),axis.text.x=element_text(angle=45,hjust=1))

Multidimentional plotting

Principal component

Selection has happened on all treated samples, but it is not clear wether the same sgRNAs are being selected. We can investigate the similarity in RPM profile by calculating the fisrt two principal components. For this we first need to generate a matrix of sgRNA x Library using the function reshape2:dcast

CountSgMat<-dcast(data=CountSg,sgRNA~Library,value.var="RPM")
head(CountSgMat)

We can now convert this dataframe into a matrix, by assinging sgRNA field to rownames.

rownames(CountSgMat)<-CountSgMat$sgRNA
CountSgMat<-select(CountSgMat,-sgRNA)
CountSgMat<-as.matrix(CountSgMat)

now I can calcualte the principal components

pca<-prcomp(CountSgMat)

This is a list of two matrices, one for the standard deviation of each PC (pca\(sdev) and one with the values of each PC (pca\)rotation).

In order to use ggplot to plot the PC, I need to convert into a dataframe and add the name of each library as a new field to the pca

pca<-as.data.frame(pca$rotation)
pca<-mutate(pca,Library=rownames(pca))

I can now plot using ggplot

ggplot(pca,aes(PC1,PC2,col=Library))+geom_point()

Now let’s make it prettier

pca$Library<-factor(pca$Library,levels=c("I1R1T0_1","I1R2T0_1","I2R1T0_1","I2R2T0_1","I1R1T3_U","I1R2T3_U","I2R1T3_U","I2R2T3_U","I1R1T3_T","I1R2T3_T","I2R1T3_T","I2R2T3_T"))
ggplot(pca,aes(PC1,PC2,col=Library))+geom_point(size=3, alpha=0.5)+
  theme(text=element_text(size=14))

NA

We clearly see that all Baseline are similar, all untreated are similar, but each treated is very different, with a very strong outlier. Can I use custome colors and add some labels?

ggplot(pca,aes(PC1,PC2,col=Library))+geom_point(size=3, alpha=0.5)+
  theme(text=element_text(size=14))+
  scale_color_manual(values=c(rep("grey",4),rep("steelblue",4),rep("tomato",4)))

NA

clustering and heatmap

We can use the same XX

Other resources

I enjoy the blog from Steven Turner “Getting Genetics Done” http://www.gettinggeneticsdone.com, which provides great example of R/bioconductor applied to genomics.

For a more classic trainign in R and bioconductor, refered to the excellent tutorial from Thomas Girk http://manuals.bioinformatics.ucr.edu/home/R_BioCondManual

Finally, there are a number of chearsheet, that you may want to keep close to you. dplyr & tidyr https://www.rstudio.com/wp-content/uploads/2015/02/data-wrangling-cheatsheet.pdf ggplot https://www.rstudio.com/wp-content/uploads/2015/03/ggplot2-cheatsheet.pdf colors https://www.nceas.ucsb.edu/~frazier/RSpatialGuides/colorPaletteCheatsheet.pdf

LS0tCnRpdGxlOiAiQ1JJU1BSIHNjcmVlbmluZyAtIFIgcHJhY3RpY2UiCm91dHB1dDogaHRtbF9ub3RlYm9vawotLS0KCkluIHRoaXMgbm90ZWJvb2ssIHdlIHdpbGwgaW50cm9kY3VlIGEgZmV3IFIgcGFja2FnZXMgdGhhdCBjYW4gYmUgaGFuZHkgdG8gbWFuaXB1bGF0ZSwgc3VtbWFyaXplIGFuZCBwbG90IHRoZSBkYXRhIGNvbWluZyBmcm9tIHRoZSBQaW5BUEwtcHkgcGlwZWxpbmUuIEltcG9ydGFudGx5LCB0aGUgbWV0aG9kb2xvZ2llcyBwcmVzZW50ZWQgaGVyZSBjYW4gYmUgYXBwbGllZCBhbmQgZXhwYW5kZWQgdG8gYW55IHJlYXNvbm5hYmx5IHNpemVkIGRhdGFzZXQuIAoKVGhlIHBhY2tnZXMgdXNlZCBhcmUgbG9hZGVkIHVzaW5nIHRoZSBmb2xsb3dpbmcgc2V0IG9mIGNvbW1hbmRzLiBUaGlzIGFzc3VtZXMgdGhlIHBhY2thZ2VzIGhhdmUgYmVlbiBpbnN0YWxsZWQgdXNpbmcgdGhlIGluc3RhbGwucGFja2FnZSBjb21tYW5kIG9yIHRoZSBSc3R1ZGlvIFRvb2xzPkluc3RhbGwgUGFja2FnZXMgbWVudS4gCgpgYGB7cn0KbGlicmFyeShkcGx5cikgIyBkYXRhIHdyYW5nbGluZyBhbmQgZGF0YWZyYW1lIG1hbmlwdWxhdGlvbgpsaWJyYXJ5KHRpZHlyKSAjIGRhdGEgd3JhbmdsaW5nIGFuZCBkYXRhZnJhbWUgbWFuaXB1bGF0aW9uCmxpYnJhcnkocmVzaGFwZTIpICMgY29udmVyc2lvbiBiZXR3ZWVuIGxvbmcgYW5kIHdpZGUgZGF0YWZyYW1lIGZvcm1hdHMKbGlicmFyeShpbmVxKSAjIGNhbGN1bGF0aW5nIGdpbmkgY29lZmZpY2llbnQgKGFuZCBvdGhlcnMpCmxpYnJhcnkoZ2dwbG90MikgIyB2ZXJzdGlsZSBwbG90dGluZyBwYWNrYWdlCmxpYnJhcnkocGhlYXRtYXApICMgY2x1c3RlcmluZyBhbmQgcGxvdHRpbmcgcHJldHR5IGhlYXRtYXBzCmBgYAoKSW4gb3JkZXIgdG8gYmVzdCBiZW5lZml0IGZyb20gdGhpcyBwcmFjdGlzZSBhbmQgdXNlIGNhc2UsIG9uZSBuZWVkcyB0byBiZSBmYW1pbGlhciB3aXRoIFIgZGF0YSBzdHJ1Y3R1cmUgYW5kIG1hbmlwdWxhdGlvbiwgbW9zdGx5IGRpZmZlcmVudGF0aW5nIHZlY3RvciwgbWF0cml4IGFuZCBkYXRhZnJhbWVzLiBBIHByaW1lciBvbiB0aGUgZGlmZmVyZW50IGRhdGEgdHlwZXMgKG51bWVyaWMsIGludGVnZXIsIGNoYXJhY3RlciwgZXRjKSBpcyBhbHNvIHVzZWZ1bC4gCgpUaGUgZGF0YSB1c2VkIGluIHRoaXMgd29ya3Nob3AgY29ycmVzcG9uZHMgdG8gYSBHRUNLTyBzY3JlZW4gaW4gYSBjYW5jZXIgY2VsbCBsaW5lLCBsb29raW5nIGZvciBlbnJpY2htZW50IG9mIHNoUk5BIGluIGNlbGxzIHRyZWF0ZWQgd2l0aCBhIGdlbm90b3hpYyBkcnVnICBmb3IgMWguIDQgbGlicmFyaWVzIGZyb20gZWFjaCBCYXNlbGluZSAoVDApLCBUcmVhdGVkIChUM19UKSBhbmQgVW50cmVhdGVkIChUM19VKSB3ZXJlIHNlcXVlbmNlZC4gSW4gdGhlIGZpcnN0IHBhcnQgd2Ugd2lsbCBzdHVkeSB0aGUgY291bnQgZmlsZXMsIGluIGEgc2Vjb25kIHNlY3Rpb24gdGhlIHNpZ25pZmljYW5jZSBhbmFseXNpcy4gCgojIERhdGEgSW1wb3J0IGFuZCBGb3JtYXR0aW5nCgpUaGUgcGlwZWxpbmUgcGxhY2VzIHRoZSBjb3VudCBkYXRhIGluIGRpZmZlcmVudCBmb2xkZXJzLiBXaGlsZSB0aGVyZSBpcyBhIHdheSB0byBoYXZlIFIgY3Jhd2wgdGhyb3VnaCBtdWxpdHBsZSBmb2xkZXJzIGFuZCBpbXBvcnQgdGhlIGRhdGEsIHdlIHdpbGwgYXNzdW1lIGhlcmUsIGZvciBjb252ZW5pZW5jZSwgdGhhdCB0aGUgY291bnRzIGZpbGVzIGhhdmUgYmVlbiBwbGFjZWQgaW4gdGhlIHNhbWUgImNvdW50cyIgZm9sZGVyIGFuZCByZW5hbWVkIGFmdGVyIHRoZSBsaWJyYXJ5IElELiBXZSB3aWxsIGltcG9ydCB0aGUgbm9uLW5vcm1hbGl6ZWQgY291bnRzLiAKCkZpUnN0IHdlIHdyaXRlIGEgZnVuY3Rpb24gdGhhdCB0YWtlcyB0aGUgZmlsZSBuYW1lIChhbmQgcGF0aCksIGltcG9ydCB0aGUgY29udGVudCBpbnRvIGEgZGF0YWZyYW1lLCBuYW1lcyB0aGUgY29sdW1ucywgYW5kIGFkZCBhIGNvbHVtbiBjb250YWluaW5nIHRoZSBuYW1lIG9mIHRoZSBmaWxlLiAKCmBgYHtyfQppbXBvcnRDb3VudHM8LWZ1bmN0aW9uKHgpewogIGY8LXJlYWQuZGVsaW0oeCxoZWFkZXI9RkFMU0UpCiAgY29sbmFtZXMoZik8LWMoInNnUk5BIiwiZ2VuZSIsICJ2YWx1ZSIpCiAgZiRMaWJyYXJ5PC14CiAgcmV0dXJuKGYpCn0KYGBgCk5vdyB3ZSBhcHBseSB0aGlzIGZ1bmN0aW9uIHRvIGEgbGlzdCBvZiBmaWxlcyBpbiB0aGUgZm9sZGVyLgoKYGBge3J9CmZpbGVzIDwtIGxpc3QuZmlsZXMocGF0aD0iLi9jb3VudHMiLHBhdHRlcm49Ii50c3YiLGZ1bGwubmFtZXMgPSBUUlVFKQpDb3VudFNnPC1zYXBwbHkoZmlsZXMsaW1wb3J0Q291bnRzLHNpbXBsaWZ5PUZBTFNFKQoKYGBgCldlIGdldCBhIGxpc3Qgb2YgZGF0YWZyYW1lcywgd2hpY2ggd2UgY29sbGFwc2UgaW50byBvbmUgZ2lhbnQgZGF0YWZyYW1lIHRvIHdoaWNoIHdlIHJlbW92ZSB0aGUgcm93bmFtZXMsIHNpbmNlIHRoZXkgYXJlIG5vdCByZWxldmFudC4KIAogCmBgYHtyfQpDb3VudFNnPC1kby5jYWxsKHJiaW5kLENvdW50U2cpCnJvd25hbWVzKENvdW50U2cpPC1OVUxMCmBgYAoKV2UgY2FuIG5vdyBlZGl0IHRoZSBkYXRhZnJhbWUgYW5kIHNlcGFyYXRlIHRoZSBsaWJyYXJ5IGZpZWxkcyBpbnRvIHN1YmZpZWxkcyB1c2luZyB0aGUgZnVuY3Rpb24gdGlkeXI6OnNlcGFyYXRlIGZ1bmN0aW9uCgoKYGBge3J9CkNvdW50U2c8LXNlcGFyYXRlKENvdW50U2csIExpYnJhcnksIHNlcD0iWy4vXSIsaW50bz1jKCJwcmUxIiwicHJlMiIsImZvbGRlciIsIkxpYnJhcnkiLCJleHQxIiwiZXh0MiIpLHJlbW92ZT1UUlVFKQoKYGBgCmFuZCB3ZSBrZWVwIG9ubHkgdGhlIGNvbHVtbiB3ZSBjYXJlIGFib3V0IHVzaW5nIGRwbHlyOjpzZWxlY3QKYGBge3J9CkNvdW50U2c8LXNlbGVjdChDb3VudFNnLHNnUk5BLGdlbmUsdmFsdWUsTGlicmFyeSkKYGBgCgpXZSBoYXZlIG5vdyBhIGxhcmdlIGRhdGFmcmFtZSBjb250YWluaW5nIGFsbCB0aGUgY291bnRzIGRhdGEKCiMgU3VtbWFyeSBTdGF0aXN0aWNzIGFuZCBOb3JtYWxpemF0aW9uCgpIZXJlIHdlIHdpbGwgdXNlIG1vc3RseSBmdW5jdGlvbnMgZnJvbSB0aGUgZHBseXIgcGFja2FnZS4gZm9yIGV4YW1wbGUgdG8gZGV0ZXJtaW5lIHRoZSB0b3RhbCBudW1iZXIgb2YgY291bnRzIGluIGVhY2ggbGlicmFyeQoKYGBge3J9CnRvdGFsQ291bnRzPC1Db3VudFNnICU+JSBncm91cF9ieShMaWJyYXJ5KSAlPiUgc3VtbWFyaXplKFRvdGFsPXN1bSh2YWx1ZSkpCnRvdGFsQ291bnRzCmBgYAoKV2UgY2FuIHRoZXJlZm9yZSBub3JtYWxpemVkIHRoZSBjb3VudHMgdG8gdGhlIHRvdGFsIG51bWJlciBvZiBjb3VudHMgaW4gZWFjaCBsaWJyYXJ5IGFuZCBtdWx0aXBseSBieSAxRTYgdG8gZ2V0IGFuIFJQTSAocmVhZCBwZXIgbWlsbGlvbikgdW5pdC4gRm9yIHRoaXMgd2UgY2FuIHVzZSB0aGUgZnVuY3Rpb24gZHBseXI6Om11dGF0ZSwgd2hpY2ggY2FuIGFzc2lnbiB0Z2hlIHJlc3VsdHMgb2YgYW4gb3BlcmF0aW9uIHRvIGEgbmV3IGNvbHVtbi4gQnV0IGJlZm9yZSB3ZSBzcGVjaWd5IHRoZSBncm91cGluZyB2YXJpYWJsZS4gCgoKYGBge3J9CkNvdW50U2c8LUNvdW50U2cgJT4lIGdyb3VwX2J5KExpYnJhcnkpICU+JSBtdXRhdGUoUlBNPXZhbHVlKjFFNi9zdW0odmFsdWUpKQpgYGAKCldlIGNhbiBub3cgdmVyaWZ5IHRoYXQgYWxsIHRoZSBsaWJyYXJpZXMgaGF2ZSB0aGUgc2FtZSBzdW0gb2YgUlBNIGFuZCBjYW4gYmUgY29tcGFyZWQKCmBgYHtyfQpDb3VudFNnICU+JSBncm91cF9ieShMaWJyYXJ5KSAlPiUgc3VtbWFyaXplKFRvdGFsPXN1bShSUE0pKQpgYGAKCgojIHBsb3R0aW5nIGN1bXVsYXRpdmUgZGlzdHJpYnV0aW9uIG9mIGNvdW50cwoKTGV0J3Mgbm93IGxvb2sgYXQgdGhlIGRpc3RyaWJ1dGlvbiBvZiBSUE0gaW4gZWFjaCBsaWJyYXJ5LiBGb3IgdGhpcywgd2UgY2FuIHVzZSB0aGUgZ2dwbG90MiBwYWNrYWdlIHRvIHBsb3QgdGhlIGN1bXVsYXRpdmUgZGlzdHJpYnV0aW9uIGZ1bmN0aW9uLiBXaGljaCBpcyBzaW1pbGFyIHRvIHRoZSBMb3JlbnogcGxvdCBnZW5lcmF0ZWQgYnkgUGluQVBMLVB5LiBMZXQncyBzdGFydCB3aXRoIG9uZSBsaWJyYXJ5LgoKCmxldCdzIHN0YXJ0IHdpdGggb25lIGxpYnJhcnkgdXNpbmcgdGhlIGRwbHlyOjpmaWx0ZXIgY29tbWFuZAoKYGBge3J9CkMxPC1maWx0ZXIoQ291bnRTZyxMaWJyYXJ5PT0iQ29udHJvbF8xIikKYGBgCgpBIHR5cGljYWwgZ2dwbG90IGNvbW1hbmQgc3RhcnRzIHdpdGggdGhlIGBnZ3Bsb3RgIGZ1Y250aW9uLCB3aGljaCBzcGVjaWZpZXMgdGhlIGRhdGFmcmFtZSBhbmQgdGhlIHZhcmlhYmxlIHRvIHVzZSBpbiB0aGUgImFlc3RoZXRpYyIgKHgsIHksIGNvbG9ycywgc2l6ZXMsIGV0YywgZXRjKS4gVGhpcyBmdW5jdGlvbiBpcyB0aGVtIG1vZGlmaWVkIGJ5IGFkZGluZyB0aGUgdHlwZSBvZiBwbG90IGRlc2lyZWQsIGhlcmUgc3RhdF9lY2RmKCkuIFRoZSBSUE0gdmFsdWVzIGFyZSBmb2xsb3dpbmcgYW4gZXhwb25lbnRpYWwgZGlzdHJpYnV0aW9uIChsb3RzIG9mIGxvdyBjb3ZlcmVkIHNnUk5BKSwgd2Ugd2lsbCB0aGVyZWZvcmUgcGxvdCB0aGVpciBsb2cxMC4gSW4gb3JkZXIgdG8gYWxzbyByZXByZXNlbnQgdGhlIHNnUk5BIHRoYXQgYXJlIG5vdCBjb3ZlcmVkIGluIHRoZSBsaWJyYXJ5IChSUE09MCksIHdlIHdpbGwgYWRkIGEgc21hbGwgbnVtYmVyIHRvIGFsbCBSUE0uIEZpbmFsbHksIHdlIHNwZWNpZmljIHRoZSBzaXplIG9mIHRoZSBwbG90IChpbiBpbmNoZXMpIGluIHRoZSBoZWFkZXIuIAoKYGBge3IgZmlnLndpZHRoPTQsIGZpZy5oZWlnaHQ9NH0KZ2dwbG90KEMxLGFlcyhsb2cxMChSUE0rMC4wMSkpKStzdGF0X2VjZGYoKQpgYGAKCldlIGNhbiBub3cgY2hhbmdlIGEgYml0IHRoZSBsYWJlbHMgYW5kIHRoZSBzaXplIG9mIHRoZSBmb250IHRvIG1ha2UgaXQgcHJldHRpZXIKCmBgYHtyIGZpZy53aWR0aD00LCBmaWcuaGVpZ2h0PTR9CmdncGxvdChDMSxhZXMobG9nMTAoUlBNKzAuMDEpKSkrc3RhdF9lY2RmKCkrCiAgeWxhYigiRnJhY3Rpb24gb2Ygc2dSTkEiKSsKICB4bGFiKCJsb2cxMChSUE0pIikrCiAgdGhlbWUodGV4dD1lbGVtZW50X3RleHQoc2l6ZT0yMCkpCmBgYAogYW5kIG5vdyB3ZSBjYW4gcGxvdCBhbGwgbGlicmFpZXMgb24gdGhlIHNhbWUgcGxvdCwgYnkgc3RhcnRpbmcgZnJvbSB0aGUgZnVsbCBkYXRhZnJhbmUgYW5kIGFkZGluZyBhIGNvbG9yIHZhcmlhYmxlIHRvIHRoZSBhZXN0aGV0aWMsIChhZG4gYWRqdXN0aW5nIHRoZSlhbmQgYWRqdXN0aW5nIHBhbmVsIHdpb2R0aCB0byBtYWtlIHJvb20gZm9yIHRoZSBsZWdlbmQpCiAKYGBge3IgZmlnLndpZHRoPTUsIGZpZy5oZWlnaHQ9NH0KZ2dwbG90KENvdW50U2csYWVzKGxvZzEwKFJQTSswLjAxKSxjb2w9TGlicmFyeSkpK3N0YXRfZWNkZigpKwogIHlsYWIoIkZyYWN0aW9uIG9mIHNnUk5BIikrCiAgeGxhYigibG9nMTAoUlBNKSIpKwogIHRoZW1lKHRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MjApKQpgYGAKCk5vdywgdGhpcyBpcyB0b28gbWFueSBsaWJyYXJpZXMgdG8gdmlzdWFsaXplIG9uIG9uZSBwbG90LCB3ZSBzaG91ZGwgYmUgYWJsZSB0byBzZXBhcmF0ZSB0aGVtIGJ5IHRpbWVwb2ludCBhbmQgY29uZGl0aW9uLiBXZSBuZWVkIHRvIGFkZCBhbm90aGVyIGZpZWxkIHRvIHRoZSBkYXRhZnJhbWUuIEZvciB0aGlzIHdlIHdpbGwgdXNlIHRoZSBmdW5jdGlvbiBkcGx5cjo6bXV0YXRlIHNlZW4gYmVmb3JlIGFuZCBwZXJmcm9tIGEgbG9naWNhbCB0ZXN0IGJhc2VkIG9uIHRoZSBuYW1lIG9mIHRoZSBsaWJyYXJ5IHVzaW5nIGdyZXBsLiBncmVwbCByZXR1cm5zIFRSVUUgaWYgdGhlIHF1ZXJpZXMgc3RyaW5nIGlzIHByZXNlbnQgaW4gdGhlIHRlc3RlZCBzdHJpbmcsIEZBTFNFIG90aGVyd2lzZS4gVGhlIGZ1bmN0aW9uIGBpZmVsc2VgIHdpbGwgYXNzaWduIHRoZSB0eXBlIHRvIHRoZSB0eXBlIHZhcmlhYmxlLCBhcyBhIHJlc3VsdCBvZiB0aGlzIHRlc3QuIAoKYGBge3J9CkNvdW50U2c8LW11dGF0ZShDb3VudFNnLHR5cGU9aWZlbHNlKGdyZXBsKCJUMCIsTGlicmFyeSksIkJhc2VsaW5lIiwiVDMiKSkKQ291bnRTZzwtbXV0YXRlKENvdW50U2csdHlwZT1pZmVsc2UoZ3JlcGwoIlQzX1QiLExpYnJhcnkpLCJ0cmVhdGVkX1QzIix0eXBlKSkKQ291bnRTZzwtbXV0YXRlKENvdW50U2csdHlwZT1pZmVsc2UoZ3JlcGwoIlQzX1UiLExpYnJhcnkpLCJ1bnRyZWF0ZWRfVDMiLHR5cGUpKQpgYGAKd2UgY2FuIHZlcmlmeSB0aGUgbmV3IGZpZWxkLCBieSBsaXN0aW5nIHRoZSB1bmlxdWUgY29sdW1ucwpgYGB7cn0KQ291bnRTZyAlPiUgc2VsZWN0KExpYnJhcnksdHlwZSkgJT4lIHVuaXF1ZSgpCmBgYAogYW5kIHdlIGNhbiBub3cgdXNlIHRoZSBmdWNudGlvbiBmYWNldF93cmFwIGluIGdncGxvdCB0byBzZXBhcmF0ZSBlYWNoIHR5cGUgb24gYSBzZXBhcmF0ZSBwYW5lbCwgaW5jcmVhc2VpbmcgdGhlIHdpZHRoIChyZXNwIGhlaWd0aCkgdG8gbWFrZSByb29tIGlmIHdlIGRpc3BsYXkgdGhlbSBpbiBhIHJvdyAocmVzcC4gY29sdW1uKQogCiAKYGBge3IgZmlnLndpZHRoPTgsIGZpZy5oZWlnaHQ9NH0KZ2dwbG90KENvdW50U2csYWVzKGxvZzEwKFJQTSswLjAxKSxjb2w9TGlicmFyeSkpK3N0YXRfZWNkZigpKwogIHlsYWIoIkZyYWN0aW9uIG9mIHNnUk5BIikrCiAgeGxhYigibG9nMTAoUlBNKSIpKwogIHRoZW1lKHRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MjApKSsKICBmYWNldF93cmFwKH50eXBlLG5jb2w9MykKYGBgCgpXZSBjYW4gY2xlYXJseSBzZWUgdGhhdCBzb21lIHNlbGVjdGlvbiBoYXBwZW5lZCBpbiB0aGUgdHJlYXRlZCBzYW1wbGUuIExldCdzIGNhbGN1bHRlIHRoZSBnaW5pIGNvZWZmaWNpZW50IGZvciBlYWNoIGxpYnJhcnkuIHRoaXMgaXMgcHJvdmlkZWQgYnkgdGhlIGZ1bmN0aW9uIGluZXE6OmluZXEKCmBgYHtyfQpnaW5pPC1Db3VudFNnICU+JSBncm91cF9ieShMaWJyYXJ5LHR5cGUpICU+JSBzdW1tYXJpemUoZ2luaT1pbmVxKFJQTSx0eXBlPSJHaW5pIikpCmdpbmkKYGBgCgpMZXQncyBwbG90IHRoZSBjb3JyZXNwb25kaW5nIGJhciBncmFwaCB1c2luZyB0aGUgZ2VvbV9iYXIgZnVuY3Rpb24uIEJlYXVzZSB0aGUgdmFsdWUgdG8gcGxvdCBpcyBkaXJlY3RseSBnaXZlbiBieSB0aGUgZ2luaSBmaWVsZCBhbmQgdGhhdCBubyBtYXRoIG9yIGNhbGN1bGF0aW9ucyBhcmUgbmVlZGVkLCB3ZSBuZWVkIHRvIHNwZWNpZmljIHN0YXQ9ImlkZW50aXR5IgoKYGBge3IgZmlnLndpZHRoPTUsIGZpZy5oZWlnaHQ9NH0KZ2dwbG90KGdpbmksYWVzKExpYnJhcnksZ2luaSxmaWxsPXR5cGUpKStnZW9tX2JhcihzdGF0PSJpZGVudGl0eSIpKwogeWxhYigiZ2luaSIpKwogIHhsYWIoIkxpYnJhcmllcyIpKwogIHRoZW1lKHRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MTQpKQpgYGAKCk5vdyB0aGlzIGlzIG5vdCBwcmV0dHksIHdlIGNhbiByb3RhdGUgdGhlIHggYXhpcyBsYWJlbHMgaW4gdGhlIHRoZW1lIHZhcmlhYmxlLiBCdXQgYmVmb3JlIHdlIG5lZWQgdG8gY2hhbmdlIHRoZSBsZXZlbHMgb2YgdGhlIE5hbWUgYW5kIHR5cGUgdmFyaWFibGUgc28gdGhhdCB0aGV5IGFyZSBub3QgcGxvdHRlZCBpbiBhbHBoYWJldGljYWwgb3JkZXIuIAoKYGBge3J9CmdpbmkkTGlicmFyeTwtZmFjdG9yKGdpbmkkTGlicmFyeSxsZXZlbHM9YygiSTFSMVQwXzEiLCJJMVIyVDBfMSIsIkkyUjFUMF8xIiwiSTJSMlQwXzEiLCJJMVIxVDNfVSIsIkkxUjJUM19VIiwiSTJSMVQzX1UiLCJJMlIyVDNfVSIsIkkxUjFUM19UIiwiSTFSMlQzX1QiLCJJMlIxVDNfVCIsIkkyUjJUM19UIikpCmBgYApOb3cgd2UgY2FuIHJlcGxvdCwgcm90YXRpZ24gdGhlIHggYXhpc3MKCmBgYHtyIGZpZy53aWR0aD01LCBmaWcuaGVpZ2h0PTR9CmdncGxvdChnaW5pLGFlcyhMaWJyYXJ5LGdpbmksZmlsbD10eXBlKSkrZ2VvbV9iYXIoc3RhdD0iaWRlbnRpdHkiKSsKIHlsYWIoImdpbmkiKSsKICB4bGFiKCJMaWJyYXJpZXMiKSsKICB0aGVtZSh0ZXh0PWVsZW1lbnRfdGV4dChzaXplPTE0KSxheGlzLnRleHQueD1lbGVtZW50X3RleHQoYW5nbGU9NDUsaGp1c3Q9MSkpCmBgYAoKIyBNdWx0aWRpbWVudGlvbmFsIHBsb3R0aW5nCgojIyBQcmluY2lwYWwgY29tcG9uZW50CgpTZWxlY3Rpb24gaGFzIGhhcHBlbmVkIG9uIGFsbCB0cmVhdGVkIHNhbXBsZXMsIGJ1dCBpdCBpcyBub3QgY2xlYXIgd2V0aGVyIHRoZSBzYW1lIHNnUk5BcyBhcmUgYmVpbmcgc2VsZWN0ZWQuIFdlIGNhbiBpbnZlc3RpZ2F0ZSB0aGUgc2ltaWxhcml0eSBpbiBSUE0gcHJvZmlsZSBieSBjYWxjdWxhdGluZyB0aGUgZmlzcnQgdHdvIHByaW5jaXBhbCBjb21wb25lbnRzLiBGb3IgdGhpcyB3ZSBmaXJzdCBuZWVkIHRvIGdlbmVyYXRlIGEgbWF0cml4IG9mIHNnUk5BIHggTGlicmFyeSB1c2luZyB0aGUgZnVuY3Rpb24gcmVzaGFwZTI6ZGNhc3QKCmBgYHtyfQpDb3VudFNnTWF0PC1kY2FzdChkYXRhPUNvdW50U2csc2dSTkF+TGlicmFyeSx2YWx1ZS52YXI9IlJQTSIpCmhlYWQoQ291bnRTZ01hdCkKYGBgCldlIGNhbiBub3cgY29udmVydCB0aGlzIGRhdGFmcmFtZSBpbnRvIGEgbWF0cml4LCBieSBhc3Npbmdpbmcgc2dSTkEgZmllbGQgdG8gcm93bmFtZXMuIApgYGB7cn0Kcm93bmFtZXMoQ291bnRTZ01hdCk8LUNvdW50U2dNYXQkc2dSTkEKQ291bnRTZ01hdDwtc2VsZWN0KENvdW50U2dNYXQsLXNnUk5BKQpDb3VudFNnTWF0PC1hcy5tYXRyaXgoQ291bnRTZ01hdCkKYGBgCgoKCm5vdyBJIGNhbiBjYWxjdWFsdGUgdGhlIHByaW5jaXBhbCBjb21wb25lbnRzCgpgYGB7cn0KcGNhPC1wcmNvbXAoQ291bnRTZ01hdCkKYGBgClRoaXMgaXMgYSBsaXN0IG9mIHR3byBtYXRyaWNlcywgb25lIGZvciB0aGUgc3RhbmRhcmQgZGV2aWF0aW9uIG9mIGVhY2ggUEMgKHBjYSRzZGV2KSBhbmQgb25lIHdpdGggdGhlIHZhbHVlcyBvZiBlYWNoIFBDIChwY2Ekcm90YXRpb24pLiAKCkluIG9yZGVyIHRvIHVzZSBnZ3Bsb3QgdG8gcGxvdCB0aGUgUEMsIEkgbmVlZCB0byBjb252ZXJ0IGludG8gYSBkYXRhZnJhbWUgYW5kIGFkZCB0aGUgbmFtZSBvZiBlYWNoIGxpYnJhcnkgYXMgYSBuZXcgZmllbGQgdG8gdGhlIHBjYQoKYGBge3J9CnBjYTwtYXMuZGF0YS5mcmFtZShwY2Ekcm90YXRpb24pCnBjYTwtbXV0YXRlKHBjYSxMaWJyYXJ5PXJvd25hbWVzKHBjYSkpCmBgYApJIGNhbiBub3cgcGxvdCB1c2luZyBnZ3Bsb3QgCgpgYGB7ciBmaWcud2lkdGg9NSwgZmlnLmhlaWdodD00fQpnZ3Bsb3QocGNhLGFlcyhQQzEsUEMyLGNvbD1MaWJyYXJ5KSkrZ2VvbV9wb2ludCgpCmBgYAoKTm93IGxldCdzIG1ha2UgaXQgcHJldHRpZXIKCmBgYHtyIGZpZy53aWR0aD01LCBmaWcuaGVpZ2h0PTR9CnBjYSRMaWJyYXJ5PC1mYWN0b3IocGNhJExpYnJhcnksbGV2ZWxzPWMoIkkxUjFUMF8xIiwiSTFSMlQwXzEiLCJJMlIxVDBfMSIsIkkyUjJUMF8xIiwiSTFSMVQzX1UiLCJJMVIyVDNfVSIsIkkyUjFUM19VIiwiSTJSMlQzX1UiLCJJMVIxVDNfVCIsIkkxUjJUM19UIiwiSTJSMVQzX1QiLCJJMlIyVDNfVCIpKQoKCmdncGxvdChwY2EsYWVzKFBDMSxQQzIsY29sPUxpYnJhcnkpKStnZW9tX3BvaW50KHNpemU9MywgYWxwaGE9MC41KSsKICB0aGVtZSh0ZXh0PWVsZW1lbnRfdGV4dChzaXplPTE0KSkKICAKYGBgCgpXZSBjbGVhcmx5IHNlZSB0aGF0IGFsbCBCYXNlbGluZSBhcmUgc2ltaWxhciwgYWxsIHVudHJlYXRlZCBhcmUgc2ltaWxhciwgYnV0IGVhY2ggdHJlYXRlZCBpcyB2ZXJ5IGRpZmZlcmVudCwgd2l0aCBhIHZlcnkgc3Ryb25nIG91dGxpZXIuIENhbiBJIHVzZSBjdXN0b21lIGNvbG9ycyBhbmQgYWRkIHNvbWUgbGFiZWxzPyAKCmBgYHtyIGZpZy53aWR0aD01LCBmaWcuaGVpZ2h0PTR9CgpnZ3Bsb3QocGNhLGFlcyhQQzEsUEMyLGNvbD1MaWJyYXJ5KSkrZ2VvbV9wb2ludChzaXplPTMsIGFscGhhPTAuNSkrCiAgdGhlbWUodGV4dD1lbGVtZW50X3RleHQoc2l6ZT0xNCkpKwogIHNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXM9YyhyZXAoImdyZXkiLDQpLHJlcCgic3RlZWxibHVlIiw0KSxyZXAoInRvbWF0byIsNCkpKQogIApgYGAKCiMjIGNsdXN0ZXJpbmcgYW5kIGhlYXRtYXAKCldlIGNhbiB1c2UgdGhlIHNhbWUgWFgKCgoKIyBPdGhlciByZXNvdXJjZXMKCkkgZW5qb3kgdGhlIGJsb2cgZnJvbSBTdGV2ZW4gVHVybmVyICJHZXR0aW5nIEdlbmV0aWNzIERvbmUiIGh0dHA6Ly93d3cuZ2V0dGluZ2dlbmV0aWNzZG9uZS5jb20sIHdoaWNoIHByb3ZpZGVzIGdyZWF0IGV4YW1wbGUgb2YgUi9iaW9jb25kdWN0b3IgYXBwbGllZCB0byBnZW5vbWljcy4gCgpGb3IgYSBtb3JlIGNsYXNzaWMgdHJhaW5pZ24gaW4gUiBhbmQgYmlvY29uZHVjdG9yLCByZWZlcmVkIHRvIHRoZSBleGNlbGxlbnQgdHV0b3JpYWwgZnJvbSBUaG9tYXMgR2lyayBodHRwOi8vbWFudWFscy5iaW9pbmZvcm1hdGljcy51Y3IuZWR1L2hvbWUvUl9CaW9Db25kTWFudWFsIAoKRmluYWxseSwgdGhlcmUgYXJlIGEgbnVtYmVyIG9mIGNoZWFyc2hlZXQsIHRoYXQgeW91IG1heSB3YW50IHRvIGtlZXAgY2xvc2UgdG8geW91LiAKZHBseXIgJiB0aWR5ciBodHRwczovL3d3dy5yc3R1ZGlvLmNvbS93cC1jb250ZW50L3VwbG9hZHMvMjAxNS8wMi9kYXRhLXdyYW5nbGluZy1jaGVhdHNoZWV0LnBkZgpnZ3Bsb3QgaHR0cHM6Ly93d3cucnN0dWRpby5jb20vd3AtY29udGVudC91cGxvYWRzLzIwMTUvMDMvZ2dwbG90Mi1jaGVhdHNoZWV0LnBkZgpjb2xvcnMgaHR0cHM6Ly93d3cubmNlYXMudWNzYi5lZHUvfmZyYXppZXIvUlNwYXRpYWxHdWlkZXMvY29sb3JQYWxldHRlQ2hlYXRzaGVldC5wZGYKCg==